Facial Recognition in Tennis
Stephanie Kobakian
18 July 2016
Project Aim
This beginning involves testing how well currently available facial recognition software performs at identifying faces of the two players in a tennis match.
Current Recognition Software
Current software has been intended for security purposes including:
- Access Points
- International Airports Passpoint points
- Watch List Screening in crowds
This presents issues as our use on Tennis Broadcasts provides multiple interchanging angles. This is unlike the intended use of a full frontal scan at an access point.
Challenges
- The multiple angles presents issues of whether the faces will be detectable
- There are many softwares available for purchase online. The most accessible during our time frame were APIs
- Softwares were intended for use beyond latent still detection, they were intended for real time detection and alerts. These softwares had capabilities far beyond what is needed
Opportunity
- We have a great opportunity to apply facial recognition software in an unusual and previously untested application
- Algorithms have certain capabilities that will allow for faces to be detected
Our Sample
105 Singles Matches from 2016 AO differing by:
- Gender, either Male or Female
- Court, one of 7 possible courts
We then took:
- 5 minute segments from the 105 Broadcast video files
- Of these segments, a still was taken at every 3 seconds
Computation Times for recognition of stills
- Skybiometry: 1.461
- Microsoft: 2.29
- Animetrics: 3.924
Examples of Application
The following slides include stills and what was recognised when the softwares were applied.
- Green: Microsoft
- Red: Animetrics
- Blue: Skybiometry
Accuracte Facial Recognition 
The software that is chosen should be able to rcognise this face even though it is not a front on angle
Accuracte Facial Recognition 
The software that is chosen should also be able to rcognise this face
Accuracte Facial Recognition 
We would hope that this angle would still allow for recognition, however it was only recognised by one software,
Accuracte Facial Recognition 
We would hope that this angle would still allow for recognition, however it was only recognised by one software
Accuracte Facial Recognition 
We would hope that this size would still allow for recognition, however it was only recognised by one software, note that the minimum pixel distance that will allow for recognition is 36
Emotion Facial Recognition 
As this is the intended future use we would need the software to detect emotions and angles such as this
Emotion Facial Recognition 
This angle should also be detected by any software we would consider for this use
Emotion Facial Recognition 
This angle should also be detected by any software we would consider for this use
Inaccuracte Facial Recognition 
Inaccuracte Facial Recognition 
Inaccuracte Facial Recognition 
Interesting Recognition 
Skybiometry was able to identify this ‘face’, created by the creases in the shirt
Interesting Recognition 
Animetrics was able to identify two faces in this image, one being a towel
Interesting Recognition 
Interesting Recognition 
Crowd Recognition 
Crowd Recognition 
Crowd Recognition 
Impractical Shots
Some shots result in difficulties when applying recognition software
Upward Angle
Birdseye View
Manual Recognition
Our Gold Standard to compare to the softwares selected
- Shiny App, using imager, shiny and shinyjs
- Scenes: live shot, person in the frame, situation, background, angle of the shot
- Faces: box, obscured, light on the face, angle of the head, glasses, or hat/visor
Still to come…
Comparisons of:
- Facial features and structures that influence recognition
- The resulting recogised areas
Questions and Recommendations